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ABSTRACT 

This study sought to determine whether the cloze 
percentage score which Bormuth identified as corresponding to the 
instructional level for intermediate grade pupils is applicable to 
more sophisticated students, Reading grade equivalent scores were 
obtained for 92 eleventh graders and 84 post-high school students on 
a standardized reading comprehension Lest. Then cloze passages on 
health education were assigned to subjects on the basis of a match 
between grade equivalent scores of the reading test and prediction of 
readability on the passages. Obtained mean cloze scores for the high 
school students were considerably higher than those for the post-high 
school students, A sampling of the cumulative records, available only 
on the high school students, for previous reading scores on other 
standardized tests yielded scores higher than those obtained in this 
study. It was hypothesized that the cloze material might have been at 
a difficulty level below these students* actual reading level. Since 
no major discrepancies ^re observed with the adult groups, the 
author concluded that it seems possible to use Bormuth* s criteria for 
determining the instructional level with adults. Tables and 
references are included. (AW) 
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Validation of the Cloze Prooedure as a Meaenre of Readability mth 
High School 5 Trade School, and College Populations 
Joe Petersons Nat Peters, and Ed Paradis^ 

A number of studies have been designed to explore the relationship 
between reacler's ability to obtain closure on a mutilated passage 
and the more traditional determinants of readability- Rather well 
documented evidence (2 ,6) has been accumulated which demonstrates that 
the cloze teehnicLUe does measure readability and can be used for ranlcing 
a given number of passages according to rellative difficulty. Recently, 
a few studies have yielded standards which might be used to determine 
whather or not a given passage is of appropriate difficulty for individual 
readers . 

These studies , by and large , are dependent upon the standards for 
comprehension which Betts (l) has determined to be appropriate for 



^Drs* Peterson and Paradis are assistant professors at the University 
of Georgia. Dr. Peters is associate director of the Language and Reading 
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the instruetional and independent reading levela , respectively 75 and 
90fi correct responses to questions based upon the selection. 

Bormuth (3,4) has reported two studies in which attempts have been 
made to identify a cloze percentage which corresponds to the instructional 
level. In his 196? article, he reported using 73 fourth and fifth grade 
nh-iidi-cn to deterralne that multiple choice Comprehension test scores 
of 73% and 90% corresponded with cloze scores of 38$ and 50$ when 
baaed on 275 word paasages centered around literature, history, and 
science. In his 1968 study, in which he is reported to place greater 
confidence, he reported that 120 four'ch, fifth, and sixth grade 
children obtained cloze scores of 44$ and 57$ to correspond with compre- 
hension scores of 75$ and 90$ with paasages t-aken from the Gray Oral 
Reading Tests, Bormuth again wrote his own questions. 

In an attempt to replicate Bormuth 'a findings, Rankin and Culhane 
(7) studied the comprehension abilities of 105 fifth grade youngsters 
reading social studies and science articles. Their procedures differed 
somewhat fx-om those uoed by Bormuth in that they constructed their 
multiple choice questions around a schema which consisted of seven 
tsrpes of questions. Including inference questions, instead of the 
straightforward measurement of only those facta presented in the selec- 
tion, In defense of hie type of questions which are literal only, Bormuth 
states that his questions are lawful and therefore replicable because 
of their hei'. "... transfomations on the syntax of the sentences in 
the passage, (4, p. 190). Regardless of this inconsistency in the develop- 
ment of questions, however, the scores obtained in the Rankin and Culhane 
study were within the ranges of the two Bormuth studies. Corresponding 
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to a multiple choice eomprehension score of 75^ was a cloise score of hl% 
as compared with Bormuth's figures of 38 and hkfa. The cloze score which 
corresponded with 90% multiple choice was 6l%. This score is higher than 
either score (50/J and 51%) reported hjr Bormuth. It is interesting to 
note that in both Bormuth's 196? study and the Rankin and Culhane 
study the subjects took the cloze test over the same passage which 
they later read and answered questions- Table. 1 presents the 



similarities of these findings. 



Insert Table 1 about here 

These studies present data which offers a frame of reference from 
which one can interpret cloze results when dealing with children in upper 
elementary school. Although the intent of the above studies was to 
present a data base from which generalizations could be drawn for all 
levels of readers, the age range sampled seems restrictive. It is the 
Intent of the present authors to present an exploratory study to deter- 
mine the feasibility of Identifying a cloze percentage which would cor- 
respond to the Betts' definiticn of instructional level for comprehen- 
sion for high school, trade school, and college students. 



Procedure 



Subjects Four different groups of subjects were used. The first 
group was 6U high school students, three classrooms of eleventh grade 
pupils from a suburban area of a northern industrial city. These students 
were enrolled in a general curriculum and were Judged to be representa- 
tive average ability students of the average high school for that city. 
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The second group was 28 high school students, eleventh graders enrolled 
in appliance repair in a regional vocational high school of the same 
northern city. The third group was 42 post-high school vocational-tech- 
nical students enrolled in a medical technology training program in a 
multi-county southern voeational-technioal school. The fourth group was 
42 upper division university students who were enrolled in a reading methods 
course in a southern university. These students were majoring in elemen- 
tary education, special education, and speech pathology. 




Data Gathering Instruments 

Two tests, one standardized and one cloze, on each subject served 
as the sources of data. The grade equivalent reading score of the sub- 
jects was obtained from the i?elson-Denny Reading Test or the Wei son Reading 
Test , comprehension subtest , depending on the estimated reading ability 
of the individual subject. 

The second sources of data were cloze tests based upon seven selec- 
tions from the area of health education which were approximately 275 
words in length. The decision was made to predict the grade level place- 
ment of these health passages with the Dale— Ohall Readability Formula 
(5) beeaus# the original validation studies had used Betts* criteria for 
comprehension on health education passages as the means for checking 
validity. That is, students reading with a given grade level equivalence 
were found able to answer correctly seventy-five percent or more of the 
questions dsked on the health education passages which were predicted 
to be at their reading level. 

The cloze passages were left with the first sentences intact and 
mutilated with an every 5th word deletion pattern. The deletions were 




represented "with a standard 15 space line. 



Answers were recorded 



directly In the cloze passages (^.e_. , no separate answer sheets were 
provided). Table 2 provides passage names ajid grads level predictions. 



Insert Table 2 about here 



Data Gathering and Analysis 

The reading grade equivalent of each subject was obtained on a 
standardized test of res/dlng Qomprehenslonj with most subjects, the 
Nelson-Penny Reading Test was used. Those subjects, however, whose 
reading; grade equivalents were determined to be low enough to be in the 
extrapolated range (i* e^. » below 9th grade ) irere ret isted with the 
Nelson Reading Test in order to obtain a more nearly acourate grade equiva- 
lent score. Since there were no grade equivalent scores on the Nelson- 
Den]^ given which were higher than l4+ to match the pasaage which was 
Identified by formula as being over 16 In difficulty, those subjeets 
who scored higher than the 50th per centile for grade 16 were CGiisidered 
to have grade l6+ proficiency in reading. 

Cloze passages were assigned to subjects on the basis of a match 
between grade equivalent scores on the reading comprehension tests and 
predictions of readability on the health education passages. Subjects 
were informed of the purpose of the experiment and were given as long 
as necessary to complete the cloze passage. Exact word replacements, 
with allowance for decipherable misspellings, were coiinted as correct. 

All cloze testing occurred within 10 days of the standardized reading 
tests. Means, standard deviations, and standard error of the mean, were 
computed for the groups individually and as a whole. 
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Results and Discussion 

The results for the four groups of eubjects measured in this study 
are reported in Table 3. 



Insert Table 3 about here 



These resultB do not appear to be in agreement ^ith those found by 
Bormuth or Rankin and Culhane as reported in Table 1. A further examina- 
tion of the data seemad warranted ^ 



An examination of mean clo^e scores presented in Table U reveals 



an inconBistency among the groups of subjects measured^ The mean scores 
for the general high school and vocational high school subjects were ^ 



Insert Table 4 about here 

considerably greater than mean seoi^es for the adtilt vocational-technical 
students and the university students, A diacussion of possible causes 
for this discrepancy follows. 

The mean scores of the two high school groups greatly exceeded the 
score for the instructional level found by Bormuth (3^4) and Rankin and 



Culhane (7) and in fact approached the independent level reported by 
these investigators 5 57 6l percent respectively. This similarity 



to the independent level combined with the vast differancas betwean tha 



high school subjects and the adult subjects suggested a search for sup- 
porting data. 

Table 5 reports tha results from the comprehanBion subteat of the 
reading achievement tests. An examination of this table indicates that 




Insert Table 5 about here 
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the adult voeational^technical subjects and the university subjects have 
mean reading scores at a level one would expect. The adult vocational 
technical subJectB were in a mediGal technology program and Judged by 
their teachers to be average high school graduates- The university 
subjects were beginning their third year in an elementary curriculum and 
while the mean score is not at grade levels it is not unusually low. 

The mean reading scores for the two high school groups are lower 
then one would expect. The general high school group was drawn from a 
school judged as average for a midwest arn urban area. A typical group of 
eleventh grade students from an average school would be expected by 
definition to have a mean grade level score of 11-00. A mean score of 
7-335 however, does not appear to be an accurate measure of a typical 
group. The mean score for the vocational high school subjecta was also 
low. I'ThxlB the vocational high school students may not t 3 rpically be 
expected to read at grade level, a mean score of 4.86 did not seem right. 

The inconsistency of reading achievement scores among the four groups 
of subjects seemed to warrant a fiarther search for information* No 
additional information was available for the general high school subjects, 
the adult vocational-technical subjects, or the imi varsity subjects. 

However, the cumulative folders for a selected group of vocational high 
school subjects were available* 

Aecordingly, the cumulative folders for the eight vocational high 
school subjects receiving the lowest reading achievement scores were 
examined* Gates— MacGlnitie Reading Tests scores from the fall of the 
school year indicated higher reading abilities than measured on the Nelson 
Reading Test * The mean grade level difference on the comprehension subtests 

was 2.86 years. While this information is from a small number of salacted 
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subjects and may involve regression to the mean 5 it does seem to question 
the amount of* confidence that cen be placed in these Welaon Beading Test 
scores* 

Additional investigation revealed that many of the high school 
subJeetB had received several standardised tests Just prior to receiving 
the Nelson Rkading Test . It is possible these subjects ^ere ^'tested 
out" and put forth a partial effort. If this was the case, one can only 
hypothesize that the cloze task intrigued the subjects and they put 
forth a maximum effort with a passage vrall below the instructional level. 
Thus, the result would be an unusually high score on the cloze task, ap- 
proaching the independent levels as defined by Bormuth or Rankin and 
Culhane * 

A second possible cause for the high school subjects- suecass could 
be their ability with the cloze procedure. If the reading grade levels 
reported in Table 5 corraot , then these subjects have likely been 
functioning In reading -material written beyond their instructional level. 
Students continually faced with material in which they can Identify 
relatively few words may develop advanced teshniques In guessing words 
from the context of the passage* This guessing skill could transfer to 
a cloze task. Thus, these subjects might be paired with passages written 
at their instructional level but advanced skill In use of context clues 
enabled them to score unusually high on the cloze task* 

The data from the high school subjects seems to raise more questions 
than it answers* Additional information seems necessary before any con- 
clusions can be made. 

The results from the adult vocational-technical subjects and the 
university subjects have been reported in Table U. labile these results 
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differ from those of the high school subjects ^ they are similar to those 
found by Bormuth (4) and Rankin and Giilhane (6). The criteria determined 
by these researchers with elementary school children seem appropriate 
for use with adults. 



Suimnary and Conclusions 

The purpose of the present study was to determine whether or not 
the close percentage score which Bormuth has identified as corresponding 
to the instructional, level for intermediate grade pupils is applicable 
to more sophisticated students. Accordingly* passages on health educa- 
tions fbe readability of which had been determined with the Dale-Chall 
readability formula, were mutilated with an every 5th word deletion 
pattern and given to IjS high school and post-high school students on 
the basis of their measured reading abllitiei. 

The results were not directly comparable with those obtained by 
Bormuth. The mean scores for the group as a whole were 50.50^ as com- 
pared with Bormuth 's finding of 

The data were examined further because of this discrepancy. Mean 
scores for the high school and vocational high school were an almost 
identical 56 . 68 $ and 56 . 69 $ while those for the vocational-technioal 
and university were almilarly grouped at 43.05$ and 44.41$, This im- 
ei^ectsd range of scores and their subsequent grouping according to the 
age of the pupils remained unexplained. A sampling of the cimiulatlve 
records* QTOilable only on the high school age students, for previous 
reading scores on other standardiaed tests yielded scores which were , 
higher than those obtained in the study. It was hypothesized that the 
cloze materiel given these students may have been at a difficulty level 
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■below their actual reading level. Directly related to this hypothesis 
is another j that the close scores of these groups would have approximated 
those of the poat-lilgh school groups had the reading levels been more 
nearly accurately obtained. 

These hypotheses are, however, yet unproved, and the possibility 
exists that the scores were accurate and the data reflect a superior 
cloze ability on the part of the general and vocational high school 
students. Additional data is warranted before conclusion can be drawn. 

Wo major diserepancies were observed or inferred with the adult 
groups. It does seem appropriate to conclude that it la possible to 
use Bormuth's criteria for determining the instructional level with 
adults . 
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Table 1 



Equivalent Cloze and Multiple-Choice Percentage Scores 
for Instructional and Independent Reading Levels 





Multiple-Choice 

Scores 


Bormuth, 1967 


Cloze Scores 
Bormuth, 1968 


Rankin and 
C-ulhane, 1969 


Instructional 


75 


38 




4l 


Reading Level 










Independent 


90 


50 


57 


61 


Reading Level 











Table 2 

Grade Level Headabillty Predictions of Health Education Passages 



Topic 


Dale-Chall Formula Ih*ediction 


Caring for Teeth 


Below 4 


The Potir Groups of Pood 


5-6 


The Middle Ear 


7-8 


Composition of Bone 


9-10 


Earache 


11-12 


Uses of Alcohol 


13-15 


Psychosomatic Disorders 


Over 16 




Table 3 



Mean Percent 


of Correct Responses 


on Cloze Passages 




Mean Cloze Score 


Standard Deviation 


Standard Error 


H 






of Mean 




50.50 


13.55 


1,02 


176 



- 


Table 


k 






Mean Cloze Percentages by Educational Training 


of Subjects 




Educational Training Mean 


Cloae Score 


Standard Deviation Standard Error 

of Mean 


U 


General High School 


56.69 


11 . 6 ? 


1 .U 6 


64 


Vocational High School 


56.68 


15.90 


3.00 


28 


Adult Vocational- 
Technieal 


U3.O5 


11.68 


1.80 


42 


University 




9.54 


1.47 


42 



Table 5 



Mean Reading Scores bjr Educational Gaining of Subjects 



Subject 


Mean Grade 
Level Score 


Standard Deviation 


StMidard Error 
of means 


■N 


General High School 


7.33 


1.26 


.16 


64 


Vocational High Elchool 


4.86 


1.67 


,31 


28 


Adult Vocational- 
Technical 


11,85 


1.73 


.27 


42 


0 

E RXC 


i 4.31 


1.68 


.26 


42 
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